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ABSTRACT 

Image processing is one of the hot research topics for the researchers where processing/transformation of the images are done. 
Image fusion is the part of image processing where the unwanted data or redundancy in the sensed information is eliminated in the 
order to reduce the energy consumption and bandwidth of the sensor nodes present in the WSN. The output obtained from image 
fusion is having very high quality information about the object then compare to images sensed by the image sensor. In this paper 
different and one of the effective image fusion techniques i.e. image fusion using Principal Component Analysis (PCA), Discrete 
Wavelet Transform (DWT), Complex Wavelets (CWT) and Laplacian Pyramids are implemented and compared. Different 
performance parameters are used for the evolution of performance. The result shows that PCA leads with greater remark in most 
of the times, sometimes DWT and CWT depending upon the decomposition levels. Worst performance by the Laplacian pyramid. 
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1. INTRODUCTION 


|mage Processing is a technique to enhance raw images received from sensors placed on satellites, space probes and 


disc@™ very 


aircrafts or pictures taken in normal day-to-day life for various applications for which they were first developed. The 
result obtained by processing is having greater quality with clear visibility of the objects present sensed image. There 
are two types of images, analog image and digital image. Now days we all preferring digital image processing as it is 
having more advantages than compare to analog image processing (Anil, 1989). Some of the fundamental steps 
involved in image processing are image representation, image preprocessing, image enhancement, image restoration, 
image analysis, image reconstruction and image data compression (Gonzalez, 1992). Image processing can be used in 
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remote sensing, medical imaging, forensic studies, textiles, material science, military, graphic arts, printing industry, 
etc. 

On the other hand image fusion is a process where unwanted data in the sensed image are removed and only 
required information which is more informative is sent to the controller or to the base station (Elmenreich, 2002). In 
other way image fusion reduces the amount of data and creates new images which are more suitable for human 
observation or further processing of images. Image fusion improves the reliability (by redundant information) and 
capability of the images (by redundant information), (Lehigh University). The image fusion can be used in target 
reorganization, object detection, military applications, medical imaging, Industrial applications, computer vision and 
robotics (Haghighat et al. 2011; Hong et al. 1999). 

Figure 1 shows the image fusion system. Here it combines two images which are having same object information 
but some of the information is not clear in image 1 but those information’s are clear in image 2 so instead of sending 
two images here fusing of the images is done, the type of fusing technology is dependent on the user’s desire. The 
fusion methodology can be averaging the images or addition of the images or subtraction of the images or minimum 
of two images or maximum of two images (Eduardo Fernandez Canga, 2002). 

Some of the popular image fusion techniques are Principal Component Analysis, Simple Averaging, Pyramid Based 
Image fusion, image fusion using wavelets, IHS Transform based image fusion, pair-wise spatial frequency matching 
and high pass filtering techniques (Deepak Kumar Sahu and Parsai, 2012). These image fusion techniques try to 
enhance the digital image without spoiling it. The fusion is performed at three different levels i.e pixel level image 
fusion, feature level image fusion and decision level image fusion. In this paper different image fusion techniques like 
Image Fusion using PCA, DWT, CWT and Laplacian Pyramid are simulated and are compared by using some standard 
performance parameters like Root Mean Square Error (RMSE), Signal to noise Ratio (SNR), Peak signal to noise ratio 
(PSNR), Mean Absolute Error (MAE), Entropy and Standard Deviation. The organization of the paper is as follows, 
section Il presents explanation about different image fusion techniques, section Ill describes the simulation of 
different image fusion techniques, section IV gives results and discussion finally section V concludes the paper. 


2. IMAGE FUSION TECHNIQUES 


Image fusion is a tool to fuse multisource images using advanced image processing techniques. Image fusion aims at 
the combination of disparate and complementary data to improve the information visible in the images as well as to 
increase the reliability of the analysis. This leads to more accurate data and improved utility. There are various 
methods that have been developed to perform image fusion. Some of the popular and known image fusion 
techniques are listed below. 


Principal Component Analysis (PCA) based Fusion 
Intensity Hue Saturation transform (IHS) based fusion 
High-pass Faltering method 

Pyramid Method 

a. Gaussian Pyramid 

b. Laplacian Pyramid 

c. Gradient Pyramid 
d 
e 
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. Morphological Pyramid 
. Ratio of Low Pass Pyramid 
5. Wavelet Transform 
a. Discrete Wavelet Transform (DWT) 
b. Stationary Wavelet Transform (SWT) 
c. Multi-wavelet Transform (MWT) 
d. Complex Wavelet Transform (CWT) 
6. Curvelet Transform 


In this paper we are concentrating on four popular image fusion techniques i.e image fusion using PCA, DWT, CWT 
and Laplacian Pyramid. 


2.1. Principal Component Analysis (PCA) 

Principal Component Analysis (PCA) is mainly used in image compression and image classification techniques. 
Principal Component Analysis (PCA) is a vector space transform often used to reduce multidimensional data sets to 
lower dimensions for analysis in other words PCA transform the number of correlated variables into uncorrelated 
variables called principal components. The main advantage of PCA is, when its data size is compressed as well as if 
dimensions are altered then there is no much loss of information at the output image (Metwalli et al. 2009; Yue-Hao 
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Figure 1 
Image Fusion System 
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et al. 2012). The fusion is achieved by weighted average of 
images to be fused. The weights for each source image are 
obtained from the Eigen vector related to the largest Eigen value 
of the covariance matrices of each source. It computes a compact 
and optimal description of the data set. The PCA does not have a 
fixed set of basis vectors like FFT, DCT and wavelet etc. and its 
basis vectors depend on the data set. PCA can also be called as 
Hotelling Transform or Karhunen-Loéve transforms (Anjali 
Malviva and Bhirud, 2009). 

In this the first principal component is taken along the 
direction of the maximum variance. The second principal 
component is forced to lie in the subspace vertical 
(perpendicular) of the first. Within this subspace, this component 
points the direction of maximum variance. The third principal 
component is taken in the maximum variance direction in the 
subspace vertical to the first two and so on (Naidu and Raul et al. 
2008). The image fusion using Principal component analysis is 
represented in Figure 2. The information flow diagram of PCA- 
based image fusion algorithm is shown in Figure 2. The input 
images (images to be fused) A (x, y) and B (x, y) are arranged in 
two column vectors and their empirical means are subtracted. 
The resulting vector has a dimension of n x 2, where n is length of 
the each image vector. Compute the eigenvector and eigenvalues 
for this resulting vector are computed and the eigenvectors 
corresponding to the larger eigenvalue obtained. The normalized 
components P1 and P2 (i.e., P1 + P2 = 1) are computed from the 
obtained eigenvector. The fused image is: |= AP1+BP2. 


2.2. Discrete Wavelet Transform (DWT) 
In discrete wavelet transform an image signal can be analyzed by 
passing it through analysis filter banks followed by decimation 
operation. This analysis filter bank, consisting of low pass and 
high pass filters at each decomposition stage is commonly used 
in image compression. When the signal is passed through this 
filter it corresponds to an averaging operation, extracts the 
courser information of signal. The high pass filter corresponds to 
a differencing operation, extracts the finer information of the 
signal. The outputs of the filter are then decimated by a factor of 
two (Bahram Jayidi, 2006). Finally, the image is split into four 
bands denoted by LL, HL, LH and HH after one-level 
decomposition. Further decompositions can be achieved by 
acting upon the LL sub band successively and the resultant image 
is split into multiple bands as shown in the figure 3. 

In images fine information content is generally found in the 
high frequencies where as the courser information content in the 


low frequencies. The wavelet transform uses its multiresolution capability to decompose the image into multiple 
frequency bands (Wang et al. 2002). First the wavelet transforms of the images are computed. The wavelet transform 
contains the low-high bands, the high-low bands and high-high bands of the image at different scales. Since large 
absolute transform coefficients correspond to sharper brightness changes, a good integration rule is to select, at 
every point in the transform domain, the coefficients whose absolute values are higher. In this way the fusion takes 
place in all the resolution levels and the more dominant features at each scale are preserved in the new 
multiresolution representation. Subsequently a new image is constructed by performing an inverse wavelet 
transform. The process of 2 Dimensional DWT based image fusion is as shown in figure 4. 
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Figure 4 
2-D DWT Image fusion 


2.3. Complex Wavelet transforms (CWT) 

The complex wavelet transform (CWT) is 
a complex-valued extension to the 
standard discrete wavelet transform (DWT). It is a 
two-dimensional wavelet transform which provides 
multi-resolution, sparse representation, and 
positive characterization of the arrangement of an 
image. The areas of the images more in focus give 
rise to larger magnitude coefficients within that 
region. A simple choose maximum scheme is used 
to produce the combined coefficient map. The 
resulting fused image is then produced by 
transforming the combined coefficient map using 
the inverse complex wavelet transform. The 
wavelet coefficient images show the orientated 
nature of the complex wavelet sub-bands 
(Kingsbury et al. 1999; Neumann and Steidl, 2003). The motivation for using the DT (Dual-Tree)-CWT (Selesnick et al. 
2005) for image fusion is its shift invariance together with its reduced over completeness when compared with the 
shift invariant discrete wavelet transforms (SIDWT). The improved directional selectivity of the DT-CWT should also 
provide improved result over the SIDWT. The complex wavelet transform was used with the filters given in designed 
for improved shift invariance. The Block diagram of 2-level Dual Tree CWT is as shown in figure 5. 


Figure 5 


2-Level Dual Tree CWT 


2.4. Laplacian Pyramid 

Image pyramids have been initially described for a multi-resolution image analysis and as a model for the binocular 
fusion in human vision. An image pyramid can be described as collection of low or band pass copies of an original 
image in which both the band limit and sample density are reduced in regular steps (Yaonan Wang et al. 2000). Burt 
and Adelson introduced the Laplacian algorithm in 1983 as a technique of image encoding (Burt and Adelson, 1983). 
However, it is not the encoding itself what is interesting for this research but the capability of multi-resolution 
pyramid transformation. A multi resolution pyramid transformation decomposes an image into multiple resolutions at 
different scales. A pyramid is a sequence of images in which each level is a filtered and sub-sampled copy of the 
predecessor. The lowest level of the pyramid has the same scale as the original image and contains the highest 
resolution information. Higher levels of the pyramid are reduced resolution and increased scale versions of the 
original image. Laplacian pyramid of an image is a set of band pass image; in which each is a band pass filtered copy of 
its predecessor. Band pass copies can be obtained by calculating the difference between low pass images at 
successive levels of a Gaussian pyramid. In Laplacian fusion method the following steps are involved: 1) Find the 
Gaussian pyramid, 2) generation of fused pyramid from Laplacian pyramid, 3) Decoding of the fused image. Fusion in 
done by considering the average values of the each pyramid level of the corresponding input images 11(x, y) and la(x, 
y). They are added to obtain fused pyramid. Fused image is obtained by expanding and then summing all the levels of 
the fused pyramid (Burt and Adelson, 1983). The Following figure 6 shows the block representation of the Laplacian 
Algorithm. 
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Figure 6 
Block Representation of the Laplacian Algorithm 


3. SIMULATION 

In this case we are considering four image fusion techniques i.e image fusion using PCA, DWT, CWT and Laplacian 
Pyramid. The techniques are simulated in MATLAB 7.8 (2009R) version. The steps required in the simulation of each 
of the techniques are mentioned below. 


3.1. Algorithm for Image Fusion using Principle Component Analysis 

Let the input images (images to be fused or source images) be arranged in two-column vectors. The steps followed to 
project this data into 2-D subspaces are (Hui Zou et al. 2004; Naidu and Raol. 2008): 

1. Organize the data into column vectors. The resulting matrix Z is of dimension 2 x n. 

2. Compute the empirical mean along each column. The mean vector Me has a dimension of 1x2. 

3. Subtract the empirical mean vector Me from each column of the data matrix Z. The resulting matrix X is of 
dimension 2 xn. 

4. Find the covariance matrix C of X that is C=XX™ mean of expectation = cov(X) 

5. Compute the eigenvectors V and Eigen value D of C and sort them by decreasing Eigen value. Both V and D are of 
dimension 2 x 2. 

6. Consider the first column of V which corresponds to larger Eigen value to compute P1 and P2 as 

R= n= 


rv iv 
At normalization P1+P2=1. 
7. Fused Image I= A(x,y)P1+B(x,y)P2 


3.2. Image Fusion using Laplacian Pyramid 

1. Compute the reduced resolution approximation of level g. of the input image A and image B. This is done by 
filtering by using 5X5 Gaussian filter and down sampling the filtered result by the factor 2. Place the resulting 
image at the level gi of the approximation pyramid. 

2. Create an estimate of the level go input image from the reduced resolution approximation of the step1.This is 
done by up sampling and filtering the generated approximation .The resulting prediction image is the will have 
the same dimension as the level go input image, which is a laplacain Pyramid . 

3. Compute the difference between the predicted image of step 2 and the input of step 1.Place the result in level go 
of the prediction residual pyramid. 

4. Take the mean of Pyramid A which is resultant of image A and Pyramid B which is resultant of image B and add 
weighted average of these two Pyramid, resulting Pyramid is the fused Pyramid of each corresponding level. 

5. The original image can be recovered exactly by expanding, then summing all the levels of the Fused Pyramids. 
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Figure 7 Figure 8 


Reference Image 1,(x,y) Source Image /1(x,y) 


Figure 9 Figure 10 


Sourcelmagel2(x,y) PCA fused image 


Figure 11 Figure 12 


DWT fused image CWT fused image 


Figure 13 


Laplacian pyramid fused image 


Where Xobs ANd Xmodei Values at time/place i. 
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Table 1 
Performance evaluation metrics to evaluate image fusion algorithm 


33.915 | 
SNR 12.863 9.962 27.276 | 28.212 Yes 
MAE 0.86 1.95 3.932 3.696 Yes 
PSNR 40.212 73.44 63.874 | 65.744 Yes 
PFE 1.147 1.87 1.376 1.38 Yes 
SD 69.043 73.40 73.935 74.63 No 
Image Average GOOD GOOD BETTER 
quality 


3.3. Image Fusion using Discrete Wavelet 
Transform 

Apply the DWT to Images A and B (geometrically registered 
images). 

1. Take the average of the approximate coefficients of two 
images repetitively and store the resultant in one matrix. 

2. Apply the binary decision map tree in order to choose the 
detail coefficients between the two images detail coefficients 
and store the result in the matrix. 

3. Apply the IDWT to get the fused image. 


3.4. Image Fusion using Complex Wavelet Transform 

1. Take Test Images 

2. Apply Dual Tree CWT to each of the test images 

3. Set the decomposition level (Here we have considered 3 
level decomposition) 

4. Combine the complex wavelet coefficients (Magnitudes) 

5. Applying inverse Dual Tree CWT (Reconstruction) 

6. Generating Final fused images. 


4. RESULTS 

Two scenes are considered here. For each scene two image 
sensors are used. Captured images from both image sensors 
are applied to all the four techniques. Six performance 
parameters are considered here namely, RMSE, SNR, MAE, 
PSNR, Standard Deviation and Image Quality to check the 
performance of each of the techniques. The results and their 
performance analysis are as represented below. 


Nomenclature: 

Xobs= Observed Values, Xmoae= Modeled Values, |, Ip |, In, Ir: 
Representation of the source, fused, LPF, HPF, and reference 
images, yi=True Value, e= Absolute Errors, fi=Prediction, hiy= 
Histogram of Fused Image. hi= Histogram of Reference 
Image, |1, l2= Input Images, M, N: sizes of the image, L= Total 
gray levels, 


Root Mean Square Error (RMSE): 


2 
DPfXone i— Xmoael, i) 
Ss = SSS 


My ral 
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Figure 14 Figure 15 


Reference Image I;(x,y) Source Image 11(x,y) 


Figure 16 Figure 17 
Sourcelmagel2(x,y) PCA fused image 


Figure 18 Figure19 
DWT fused image CWT fused image 


Figure 20 


Laplacian pyramid fused image 
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Mean Absolute Error (MAE): 


QD 1 D 
MAE= => li —y,l= -) leit 
t—2 


=i 


Peak Signal-to-Noise Ratio (PSNR): 
2 
PSNR = 201g, ~ = 


aba dpa! CLi-hiiy* 


STANDARD DEVATION (SD): 
richie il LLCS 
o= JEL — — Hair 


Where f= ELI « hif 


4.1. Scene 1 

In this case a cat image is taken (Figures 7-13; 
Table 1). Here one original image (Reference 
Image) and two source images are taken and are 
applied to all four techniques and output image 
i.e fused image is sometimes compared with the 
reference image for performance metric 
calculation. Finally a table is drawn which gives 
all the information regarding the techniques 
performance and image quality. 


4.2. Scene 2 

The above said operation is repeated here also 
and finally a table is drawn to compare all four 
image fusion techniques (Figures 14-20; Table 2). 
The performance of the image fusion algorithms 
could be estimated precisely when the ground 
truth (without reference) image is not available. 
The metrics showed in the table 1, table 2 
bolded font are better among others. 


5. CONCLUSION 

This paper has demonstrated the potential of 
image fusion technique as a tool for improving 
the quality of the sensed images. All techniques 
try to reduce the data size without affecting the 
important content in the sensed images also 
gives high quality information regarding the 
objects present in the images. Here mainly four 
image fusion techniques have been simulated 
and compared i.e. image fusion using DWT, 
CWT, PCA and Laplacian Pyramid. These 
techniques are simulated with and without 
reference image. The results show that the 
performance by the Laplacian pyramid is worst 
in all the cases and the performance of PCA is 
very good for most of the cases, some other 
times DWT performance hold good. CWT’s 
performance was better then compare to DWT, 
PCA and Laplacian Pyramid if decomposition 
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Table 2 


Performance evaluation metrics to evaluate image fusion algorithm 


Laplacian PCA DWT cwT Betered original 
image 
RMSE 44.72 40.88 77.48 73.375 Yes 
SNR 10.84 5.96 22.33 22.826 Yes 
MAE 1.016 3.47 6.67 6.423 Yes 
PSNR 38.74 67.824 56.59 57.581 Yes 
PFE 2.047 3.954 2.564 2.41 Yes 
SD 52.36 54.662 56.59 56.96 No 
Image quality Average BETTER GOOD GOOD 
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